1. Field of the Invention
The present invention relates to process for determining a mask for changing the brightness profile of a photographic print or copy. Furthermore, the invention relates to a process for changing the brightness profile during the manufacture of a print or copy by way of the mask determined in accordance with the invention. Finally, the invention relates to an optimal printer and a digital printer which use the mask determined in accordance with invention.
2. Background Information
The term xe2x80x9cphotographyxe2x80x9d, as used herein, refers to the (permanent) recording of images produced by electromagnetic radiation (especially light) by means suited herefor (for example, a camera with film, a digital camera with CCD-chip, a film camera, a video camera, and the like).
The field of the present invention relates to the computer and/or optical processing of the photographic image information which represents the picture produced. The photographic image information is thereby obtained, for example, by scanning a photographic master (for example a film) and converted into digital image data. The photographic image information can also be already present in digital format. It can then be stored, for example, electronically (for example, CD-ROM, DVD), can be accessible through a network (for example LAN or Internet) or can be recorded, for example, with a digital camera.
Photographic image data can be copied such as conventional image data onto a physical medium. For example, photographic paper serves as physical medium onto which photographic films are exposed. The copying can also be carried out by way of a digital printer, which is controlled by the photographic image data, and onto normal paper or photographic paper. However, the quality of the thereby achieved photographic prints is often unsatisfactory.
An essential reason for this is that the dynamics of the luminescence included in the image information to be copied cannot be transposed onto the copying medium. The reason for that is, for example, that photographic paper compared to a photographic film has a highly reduced dynamic luminescence range. But also the dynamic luminescence region which can be illustrated with a printer is smaller than the typical dynamic luminescence range of photographic image information.
The above-mentioned disadvantages encountered during the physical realization of a photographic copy have as a result that the subjectively observed brightness profile during observation of the copy is negatively distinguished from the subjectively observed brightness profile of the photographic image information underlying of the copy. This means that the picture feel is not optimal for an average observer during observation of the copy.
For example, brightness differences which often are still distinguishable for an observer of the original photographic image information are no longer distinguishable for an observer of the photographic copy when those brightness differences are found in a dark or very bright region of the photographic copy.
In order to overcome this disadvantage, masks were proposed which cause a brightening or darkening of certain regions of the copy. Such masks are described, for example, in DE 19703063 A1 or in DE 4040498 A1. A calculated mask is used therein to control an LCD matrix. This LCD matrix is found during the exposure of a photographic film onto a photographic paper in the therefore required optical copying light path and changes the brightness of the manufactured copy by region. Image data obtained from the original (photographic) image information are exclusively used for the calculation of the masks.
It is a disadvantage of the previous processes, that photographic errors included in the original photographic image information, such as color errors or errors in the brightness distribution enter into the calculation of the mask. The photographic errors can be caused, for example, by errors during photography (underexposure or overexposure), specific film properties (color shifts caused by the film) or by the spectral sensitivity of the CCDs of the digital camera used.
However, when the photographic errors enter into the calculation of the matrix, no optimal manipulation of the brightness profile of the desired photographic copy can be achieved.
It is accordingly an object of the invention to optimize the calculation of a mask for the change of a brightness profile of a photographic copy in such a way that negative effects of photographic errors are suppressed.
A mask is calculated with the process in accordance with the invention which is used for changing the brightness profile of a photographic copy.
According to the process, photographic image data are first derived from photographic image information, which includes the image information to be copied, whereby preferably a reduction (associated with loss) of the image information content, especially the image resolution, takes place. The photographic image information can include more image information than is copied, for example, the photographic image information can represent a sequence of images (for example, a film with several individual pictures or a recording sequence of a digital camera), whereby in accordance with the invention the brightness profile of the copy of one of these images is changed or the mask required therefor is determined. The image information corresponding to the remaining images can be used in accordance with invention for determination of the brightness mask (for example by way of complete film analysis). Also, the image information of a whole image can be used, when only a section of the image is to be copied. The photographic image information can be, for example, a photographic film (complete film, individual picture). The photographic image data are preferably digitalized during derivation from the photographic image information and can be derived, for example, by scanning (for example of the complete film or of only an individual picture or a section thereof) and alternatively during further processing (for example data reduction and/or smoothing). However, the photographic image information can also be present in digital form, for example, when it originates from a digital camera or is transferred through a network, for example. The photographic image data can then be derived from these digital data, in that, for example, the digitally available photographic image information is taken over 1:1. However, an especially loss associated data reduction, especially a resolution reduction, and/or smoothing is carried out in order to minimize the subsequent calculation effort.
Overall it is preferred that the number of data which include the derived photographic image data (especially per individual picture), is about 10,000 or less, preferably less than 2000 and especially preferably about 1000 (or less). Good results can be achieved in practical use even with such a reduced data volume. The calculation effort is simultaneously minimized. Especially for the generation of photographic image data by scanning an original film, one can choose an only coarse scanning from the start so that the time required for the scanning process is significantly reduced compared to a fine scanning (as is common in the prior art) and the number of the photographic image data is at the same time reduced to the desired amount.
The reduction has the further advantage that finer details of the photographic image information (of the photographic original or the scene illustrated thereby) have no influence on the masking. The already mentioned smoothing can also be of use for this purpose, whereby, for example, the image data which are reduced during the derivation from the image information to a desired number (for example in the form of a reduced image matrix) are subjected to a low pass filtering. It is advantageous in the case of color photo image data (for example a colored image matrix) to transform the color values into a color area wherein one of the axes describes the brightness of the image points. If the color information is not used in the further processing (for example for the analysis), it is also sufficient to calculate gray values from the color values (for example, by weighted averaging), which characterize the brightness of individual image points. The reduction and a matrix transformation connected therewith is described in detail in the EP 0475897. The disclosure of the reference EP 0475897 A1 is herewith incorporated into the present disclosure.
In accordance with the invention, the photographic image data which are preferably present in digital form are corrected in order to obtain optimized image data. Preferably, only that part of the photographic image data is corrected and converted into optimized image data, which corresponds to image information to be copied or to a portion to be copied. During the correction, errors are considered which were generated during the photographic recording of the image information. Optimized image data are thereby obtained which at least approximate the image data which are derived from an ideal photographic image information, which represents the photographically recorded object without error, which means corresponding to reality. Error-free recording means especially that the photographically recorded image information is free of color change. This means that the colors do not change upon a variation of the photographically recorded light amount and with a constant spectrum, so that no color change is present. Especially photographic films exhibit color change.
A correction of the photographic image data can be carried out on the basis of an analysis of an image sequence, as described or for example, in EP 0586773, which is herewith incorporated into the present description. The analysis of a whole film (xe2x80x9ccomplete film analysisxe2x80x9d) is described in the EP 0586773. For example, the color change of a film is derived from the analysis results. The result of the analysis is then used for the correction of the color dependent exposure.
If an image sequence analysis (for example, complete film analysis) is carried out in accordance with invention, in order to recognize the errors which were generated during the photographic recording of the image information and to be able to take them into consideration, the photographic image data are derived from image information which includes more than one individual picture. Then, for example, the color values are analyzed, especially statistically analyzed. A color area standardization (see also EP 0586773) is then carried out for correction depending on the analysis results. This is preferably achieved by a translation and rotation of the color area. The translation thereby takes into consideration the mean coloring and density of the actual film and the rotation its exposure dependent color change (xe2x80x9ctippingxe2x80x9d). After standardization of the color area (i.e. the correction process), the image data are largely cleared of film based distortions (i.e. errors in the photographic image recording), so that, for example, ideally the same scene, when photographed with different films, always leads to the same optimized image data. An image sequence analysis, especially with color area standardization derived therefrom, is important, since a large number of film products are on the market with sometimes significantly different image recording properties. This is also the case for the image recording properties of the different digital cameras.
In accordance with invention, alternatively or additionally other analysis types can also be selected apart from an image sequence analysis, in order to recognize the errors during image recording and to then take them into consideration for the correction. For example, one can revert back to stored data which include typical image recording properties of different films or digital cameras. The selection amongst different stored data can, for example, be carried out by reference to information, which was additionally stored by the image information carrier (for example, film cartridge or diskette) apart from the image information and, for example, regarding the film type or camera type or regarding the recording situation (artificial light or daylight) and which is also stored during the detection (recording) of the photographic image information (for example, during photography) (for example, data which are stored according to the Advanced Photo System or APS).
With respect to the consideration of film specific data, reference is again made to EP 0586773 A1.
In addition to or in the alternative to the above analysis methods, an individual image analysis can be carried out. This means that the analysis is supported by the photographic or already optimized image data of an individual image which, for example, at least essentially correspond to the image information to be recorded or include the image information to be recorded as an image portion. If only the portion of an image is to be copied (for example, panoramic image with APS), the analysis can be based, for example, on only the image data which correspond to the portion, or in addition on image data which lie outside the portion, but are part of the recorded individual image. The image attributes of an individual image are those properties of the image information representing an individual image, which can be extracted by individual image analysis from the photographic image data derived therefrom. These image attributes describe especially those attributes which especially influence the picture feel (the sensory perception). With such an analysis, especially the coloring, the contrast, the exposure as well as the color, brightness and contrast profiles are determined and examined. Furthermore, the analysis is carried out especially for the recognition of color and/or brightness errors. Also, tipping phenomena (color differences between bright areas and shadows) should be recognized.
Preferably, the analysis of the photographic image data is used for classification of the image data. This means the photographic image data are assigned to a specific class of images. Possible image classes are, for example, counter light pictures, flashlight pictures, artificial light pictures, etc. However, classes can also be used which cannot be translated directly into photographic terms in the art, in any event, the goal of the classification is the recognition of images with similar image attributes, especially with respect to contrast, exposure, coloring, etc.
Preferably, the individual attributes are combined into a N-dimensional attribute vector for the classification. The N-dimensional attribute area is (for example, by way of a clustering process) divided into continuous regions, or by each region corresponds to a class. The attribute vectors which fall within the same region originate from image information with similar image properties and are assigned to the same class.
The individual image analysis can be carried out, for example, on the basis of the photographic image data, in order to correct the photographic image data and to achieve optimized image data or to participate in their determination, for example, depending on the classification which took place. However, the individual image analysis is preferably carried out on the basis of the optimized image data. Preferably, the correction of the photographic image data and the determination of the optimized image data is in this case carried out on the basis of an image sequence analysis or on the basis of stored typical image recording data (or a combination thereof, see above).
The results of the individual image analysis are preferably she used to modify the optimized image data. This modification is preferably carried out in dependence of the classification result. The image attributes obtained from the individual image analysis, or the classification result can alternatively or additionally also enter into the determination of the brightness mask from the optimized image data and thereby modify the values of the brightness mask. The above-mentioned modifications are preferably carried out on the basis of the image attributes of an individual image, which were extracted from the optimized image data. However, they can also be carried out on the basis of other attributes of the optimized image data, for example, attributes of an image sequence. This is especially then advantageous when an image series (for example, with sports pictures) was recorded or when the photographic image information was recorded, for example, with a video camera. A uniform copy quality can be achieved in this manner for the images of a series.
The correction of the photographic image data can be carried out in several ways, for example, pixel by pixel or by blocks using a correction function. However, a correction transformation is preferably carried out, whereby the fields recognized, for example, by the analysis, are considered during the image recording or at least approximately compensated.
A translation and a rotation of the color area is preferably carried out with this correction transformation. Optimized image data are obtained with this correction transformation which is applied to the photographic image data. If a copy is produced on the basis of such optimized image data, that copy is more aesthetically pleasing to an observer than a copy based on the photographic image data. The optimized image data therefore have optimized image feel properties. Especially, irritating errors with respect to contrast, exposure and/or coloring are reduced. The correction transformation or a correction function can be optimized for example, with respect to these errors in a stepwise, especially iterative process consisting of copying, analysis of the copy, modification of the correction transformation, newly copying, etc.
In accordance with the invention the number of the optimized image data used as the basis for the mask calculation is preferably significantly reduced compared to the optimized image data eventually used as the basis for a photographic copy made with a digital printer. The former optimized image data can be derived from the latter optimized image data, for example, by a resolution reduction. However, the optimized data for the mask calculation are in this case derived from an already strongly reduced number of data, in accordance with invention.
Based on the optimized image data (for the mask calculation) which have been cleared of photographic errors or wherein the photographic errors have been at least minimized, a so-called brightness mask is then produced. This brightness mask includes values by which the brightness profile (the brightness distribution) of a photographic copy of the (original) photographic image information (for example, a film) is to be influenced.
The matrix so obtained can be, for example, stored or transferred for further processing to a device (for example, an optical or digital printer), which produces a copy of the photographic image information.
If the photographic image information is present in physical form, for example, as photographic film, the values of the brightness mask can be used for control of a light density control device, for example an LCD matrix. They can be especially used for control of the LCD device described in the European Application with the application No. 98115693.8, the disclosure of which is herewith incorporated and a copy of which is attached.
The values of the brightness mask are, as already mentioned, determined from the optimized image data. One preferably proceeds in such a way that first the gray values corresponding to these image data are determined from the optimized image data and then the values for the brightness mask are determined from those gray values.
The data processed by the process in accordance with invention are preferably arranged in a matrix format and processed by transformation of the matrices. Accordingly, the brightness mask is preferably also arranged as a matrix.
The values of the brightness mask are preferably determined in such a way that upon use of one region of the mask the brightness profile (the brightness distribution) of the photographic copy of that region of the photographic image information is influenced which also entered into the calculation of the mask region. Especially, regions in the copy with average brightness should not be influenced or only marginally influenced. Furthermore, regions of low brightness should be brightened (or remain unchanged) and/or regions with high brightness should be darkened (or remain unchanged). According to a preferred process, the brightening of dark regions is made dependent on the degree of the darkness so that the brightening in very dark regions is again attenuated or no longer carried out. Accordingly, the darkening of bright regions is preferably carried out such that the darkening in very bright regions is attenuated or possibly even completely left out.
In the just described exemplary embodiment of the brightness mask the change of the brightness profile is thus carried out depending on the brightness of the image data to be copied. More precisely, according to the present invention, the elements of the brightness mask are calculated from the optimized image data and especially from the corresponding gray values of the optimized image data. The calculated values are preferably a function of the optimized image data or the corresponding gray values. The function is preferably nonlinear. The function can be present, for example, in the form of a table or it can be realized as a programmed, partly linear function.
The function is preferably modeled so that it has the above described effect on the brightness profile of the photographic copy, for example, the brightening of dark regions at the darkening of bright regions. The degree of brightening or darkening is thereby preferably limited in such a way that upon copying or reproduction of the image no visible or irritating loss in total contrast occurs. Therefore, the already obtained analysis results or especially information on the contrast in regions of the image and/or the overall contrast preferably enter into the modeling of the function or the calculation of the mask elements. This information is obtained, for example, by the above described analysis which determines the characteristics of the photographic image data. In addition, or alternatively, a separate analysis of the optimized image data can be carried out from which the characteristics of the optimized image data are then derived which then enter into the calculation of the mask elements.
Furthermore, the degree of brightening or darkening is preferably limited so that the visibility of the noise, for example, the granularity, if the photographic image information is a negative film, is not excessively increased in the dark and or bright portions of the image. In order to determine the degree of limitation of the brightening or darkening, one can again go back to characteristics which were determined during the analysis of the photographic image data or during a separate analysis of the optimized image data (for example, overexposed, underexposed, etc.). Furthermore, information on the film type (for example, ASA No. regarding the granularity) or the photograph situation (for example, taken with flash) can enter into the determination of the function.
During the realization of the function, the limiting of the brightening or darkening can be carried out, for example, by setting a maximum value. The maximum value in turn can be a function of the specific characteristics of the photographic image information (for example, film type) or the specific characteristics of the photographic image data or the optimized image data (for example, total contrast).
The function is preferably modeled in such a way that no distortion due to image unrelated brightness jumps occurs during the masking. This means that the function to be used is preferably constant and that the slope of the function cannot be too large. It can be advantageous for this purpose to subject the brightness mask to a smoothing or to subject the matrix underlying the brightness mask to a low pass filtering.
As already mentioned above, specific characteristics or otherwise recorded characteristics or information (for example, data stored in addition to the image information during the image recording, especially data regarding the photograph situation or, for example, the camera type of a digital camera or the film type) can enter into the determination of a best suited function by the analysis of the image data. It is thereby advantageous to assign the different characteristics to different classes depending on the degree of their distinction. The best suited function is then again assigned to each class. The class formation and the function modeling for each class can then be refined with increasing experience level, for example, by way of an expert system.
Apart from the consideration of global properties, which are expressed in the image classes, it is also advantageous to consider local properties in the vicinity of a masking point. As described further below, a local control of the masking strength is preferably carried out herefor.
A local control of the masking strength is necessary, because, for example, it can lead to irritating effects when the brightness mask is superimposed with the image information used for the copy. This applies especially where the brightness profile of the image information to be copied has large jumps. For example, if a brightness step with a steep step flank of the image information to be copied is superimposed with an inverse brightness step with flat step flank of the mask, an edge superelevation can occur both in the positive as well as the negative direction. This means that at a step transition from dark to bright initially a normally dark value is present which then shortly before the step further darkens and then immediately after the step strongly brightens. The strong brightening is shortly thereafter reduced again to a normal bright value. Because of the low dynamic range of the photographic paper, the stronger darkening is normally not visible in contrast to the stronger brightening. For this reason, visible edge superelevation effects are preferably moved into such regions wherein an edge superelevation is not conspicuous for the observer. In the above described case, a mask would be structured such that an edge superelevation occurs only in the dark region (albeit stronger) and that no edge superelevation occurs in the bright region. The position of the step flank of the brightness mask is thereby changed in direction towards the darker region of the brightness step to be copied until it is completely located in the dark region.
In order to avoid undesired effects during the superimposition, for example, as are described above, the masking strength is controlled with local resolution in that preferably the values of the elements of the brightness mask are changed. The control is thereby preferably carried out such that the values in a local control region which includes an actual mask element and further mask elements are made the basis of the calculation from which a new value for the actual mask elements results.
Different nominal values or functions can be used during the calculation for the local control the results of which can be, for example, weighted and combined with one another. For example, the maximum value determination, the minimum value determination, the main value determination, the mean determination can be used as functions which are then preferably applied to the values in the mentioned manipulation region. However, the function for the determination of the standard deviation, of the detail contrast, etc. can also be used. If one uses only, for example, a minimum function, this results in the xe2x80x9ccarefulxe2x80x9d masking of the regions with higher mask values, which is equal to the flank displacement described above by example. In accordance with the invention, characteristics of the image recording (for example, film type, ASA number, etc.) and/or specific characteristics of the photographic image data and/or the optimized image data also enter into the determination of the type of the manipulation or the setting of the above-mentioned weighting during the manipulation of the values of the brightness mask. One can in this case also classify or go back to an already carried out classification. As already mentioned above, the manipulation functions (maximum function, minimum function, etc.) can be differently weighted during the determination of a new mask value. The structure of the weights can thereby be made dependent, for example, from the class specified.
The manipulation and thereby, for example, the structure of the weights can however also be made dependent from the properties (characteristics) of the different local regions of an individual image. Expressed otherwise, values of the brightness mask which concern regions of the copy with different characteristics or properties are differently manipulated. One property of the region represents, for example, it""s information content (xe2x80x9cenergy densityxe2x80x9d). For example, information rich image regions typically have strong light/dark variations. Preferably, the manipulation is carried out such a way that information poor image regions are not masked or masked less strongly than information rich image regions. Image regions with different properties or attributes are recognized, for example, by way of a xe2x80x9clocal analysisxe2x80x9d with which locally resolved regions of an image can be analyzed. This local analysis is based on, for example, a locally resolved light/dark analysis, and analysis of the light/dark variations or the contrast variations. The manipulation is carried out preferably locally differently depending on the local properties or characteristics of the image, especially a value of a brightness mask is manipulated depending on the characteristics (image characteristics) of an image region locally assigned to the value.
The local analysis/manipulation itself can be based on an analysis/manipulation of the photographic image data, optimized image data or the values of the brightness mask itself. Especially in the latter case, the analysis is preferably an integrated part of the calculation of a manipulated (modified) brightness mask (see further below, for example, the use of the standard deviation function). The local manipulation (modification) is preferably carried out such that the influencing of the brightness profile is locally xe2x80x9cdosedxe2x80x9d or varied so that locally a more or less strong influencing of the brightness profile results and depending on the locally assigned image attributes (attributes of the locally assigned image region).
To superimpose the brightness mask with the image information to be copied, it can be desired that the mask has a preselected number of elements. If the mask is present in matrix form, the matrix can be, for example, reduced or enlarged, in order to achieve an adaptation to the desired number or to a desired number of lines and columns.
Adaptation to a preselected number is especially then advantageous when a light control arrangement, for example, an LCD matrix, is controlled with the mask values during the exposure of a film onto a photographic paper. Such an LCD matrix has a preselected number of elements.
In the case in which the number of the mask elements is to be adapted, the determination of the brightness mask is preferably carried to in two steps. First, a first mask is calculated, which is to cause a brightening or darkening. This mask is preferably a matrix with the same dimensions as a matrix in which the optimized image data are illustrated. A second matrix with a fitting number of elements and with the desired number of lines and columns is then determined from the first mask. If one considers, for example, an actual element of the first mask, which is located in an assigned region, the values of the elements in this region are used as a basis for the manipulation of the value of the actual element and the value which results from the manipulation is assigned to that element of the second mask, which is assigned to the actual element of the first mask.
When, as discussed above, the local properties or actual units of an image are analyzed and if the manipulation is locally different depending on the local properties, the local properties or attributes of that region of the first mask are preferably considered for the calculation of each element of the second mask which is associated with that element. This applies especially when the first as well as the second mask are expressed in matrix format.
In the case of an optical masking, an optical mask (for example, a liquid crystal matrix or light control arrangement) is controlled with the values of the brightness mask calculated preferably in two steps (first mask and second mask). The optical mask is thereby preferably positioned at a certain distance from the plane of the original so that an unfocused image of the mask (unfocused overlay) results. The dimensions of the brightness mask are preferably selected to be equal to or at least about the same as the number of the lines and columns of the optical mask matrix.
In the different steps of the calculation of the brightness mask, one preferably insures that the geometrical assignment of the matrix points or mask points remains in tact. This means especially that neighboring mask values of a mask value can be traced back to neighboring regions of the photographic image information underlying the mask value (while maintaining the orientation) so that the brightness mask and the image information to be copied are properly superimposed in the optical system.
In general, the image information to be copied can be the same as the photographic image information or arises therefrom through processing. In an optical system, the processing can be carried out, for example, by manipulation of the exposure through color filters, so that the (original) photographic image information (information of the photographic film) differs after the filtering from the film information to be copied, which is then superimposed with the mask.
This applies correspondingly for superimposition in a processor. There, the photographic image information is often digitally present in high-resolution. This digital image information is, as already mentioned, preferably reduced in a first step of the process in accordance with the invention for the determination of the brightness mask (derivation of the photographic image data). However, for the determination of the image information to be copied no reduction or preferably only a small reduction is carried out. But preferably the processing step is a correction transformation (see EP 0586773 A1). The image information obtained after the correction transformation is then the image information to be copied, which is to be superimposed with the brightness mask (in a processor).
The masking in a processor is preferably carried out digitally or electronically. The brightness mask (in the two-step process the second mask) is thereby transformed by way of one of the known interpolation processes (for example, xe2x80x9cimage resamplingxe2x80x9d, xe2x80x9cimage resizingxe2x80x9d, xe2x80x9cup samplingxe2x80x9d. . . ) into an image with the resolution of the image information to be copied, preferably smoothed by way of a low pass filtering (preferably during or after the interpolation) and subsequently superimposed in a processor with the original image matrix. A preselected connection function is selected, for example, simple addition, for the superimposition in the processor. If the mentioned smoothing is carried out, this process corresponds to the unfocused projection in the case of the optical masking.
As mentioned above, the brightness mask is preferably derived from the gray values, which are in turn determined from the optimized image data. However, the brightness mask can also be calculated respectively separately for the processing of image information of different color to be copied (for example, for the red, green and blue exposure). Those brightness values of the optimized image data which are assigned to a specific color are thereby preferably used for the calculation of the brightness matrix intended for the corresponding color. If only one mask is used, the color information included in the optimized image data can also be taken into consideration.